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A SCIENCE-TECHNOLOGY-SOCIETY (STS) CRITICAL 
ISSUES COURSE FOR GENERAL STUDIES NATURAL 
SCIENCE CREDIT: THE EVALUATION OF A PILOT 

OFFERING 1 



James A. Rye^ Peter A. Rubba Leonard Waks 

Penn State University Penn State University Perai State University 

This paper describes an evaluation by two science educators of the pilot offering of an 
interdiscipUnary critical issues course in Science, Technology, and Society (STS), developed through a 
project funded by the National Science Foundation (NSF). The course was designed to fulfiU the natural 
science general degree requirements at a major university, and in so doing, responded strongly to the 
reformation call issued by The Study Group of the American Association for the Advancement of Science 
(AAAS) on General Education and the Sciences: Radical reform of the natural science general education 
component of the undergraduate curriculum is in order (AAAS Study Group, 1990). 

The AAAS Study Group makes clear that science is a liberal art and science instruction that 
satisfies general education requirements, must foster an understanding of concepts unifying the sciences 
with other discipUnes as well as interactions of science with technology and society. Critics contend that 
postsecondary education in the United States generally has failed to provide college graduates with the 
science and technology background they need to be liberally educated, and as responsible citizens and 
workers, serve a technological and informational society (Fol: & Roy, 1991; Hetherington, Miller, 
Sperling, & Reich, 1989). Science instruction pays insufficient attention to the applications of science in 
society and permits ignorance of science, technology, and society interactions (Bernstein, 1993; Ellis & 
Ellis, 1991). Hurd (1992,1993) and Ramsey (1993) provide strong support for utilizing the STS 
approach in the reconceptualization of science teaching and in providing for the scientific literacy needs ol 
a general education. Ellis and EUis believe that all Uberally educated citizens should be "lay scientists" as 
evidenced by their ability to understand and make informed decisions about societal issues dominated by 
science and technology. 
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, of NSF. 
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Background 

The project "Cntical Issues in Science, Technology and Society: An Integrated Course Sequence " 
(NSF-DlJE-9254108) was awarded to the STS Program at a major university in the Summer of 1993 by 
the NSF Undergraduate Course and Curriculum Development Program. The STS program is 
interdisciplinary, housed in three of the university's Colleges (Engineering, Agriculture, and Earth and 
Mineral Sciences). The primary mission of this project was to develop an interdisciplinary natural science 
general studies education course that focused on critical STS issues. This course was to be integrated with 
an exisiting social science general studies education course (three credits) at the university and become the 
second half of a sbc-credit critical issues course sequence taken over two semesters. The social science 
general studies course is an STS inteixiisciplinary course that emphasizes (a) how science and technology 
are stimulated and guided by social institutions and values, (b) the interaction of science and technology 
and their impact on our lifestyles, and (c) that responsible individuals can exert beneficial influences on 
technological developments and their social effects. As such, this social science general studies course 
v/ould provide the STS "foundations" for the investigation of the critical issues targeted in the natural 
science general studies course, hereafter known as STS 251— the actual course number was different. 

The issues proposed as foci for the STS 251 course were global or "greenhouse" warming (GW), 
energy options (EO) for the future, and biodiversity conservation (BC). These issues were chosen because 
they present a breadth of interrelated scientific content, are politically interdependent, and are socially 
urgent on a global scale. They provided a personal and societ-! context for the learning of science. 
Additionally, faculty who commanded world-class s. and technological expertise relative to these 
issues were available at the university, and interested in collaborating as an interdiscipUnary team of 
instructors for the course. Ultimately, the goal of the project was to disseminate this six-credit critical 
issues course seqence as a model within and outside the university's system. 
STS 2S1 Cnnrsp Devp lnpnipnt and Instruction 

Beginning in the Summer of 1993 and through the Fall Semester of 1993, a group of faculty from 
the STS Program who would teach STS 251 developed the pilot version of the course. This group was 
headed by the Director of the STS Program and the professor who authored the NSF proposal, and 
included instructors and a graduate assistant from the Departments of Geography, Fuel Science, and 
Entomology. The course evaluators were invited to sit in on the faculty meetings that guided course 
development; such involvement contributed to the prolonged engagement (Erlandson, Harris, Skipper, & 
AUen, 1993) the evaluators had with the course and informed the evaluation design for the pilot offering of 
the course. At these meetings, the evaluators were viewed as resources on pedagogy and were asked to 
provide the course developers (i.e., the instructors) with references and two workshops that dealt with 
current reform thrusts in science teaching, including authentic assessment, concept mapping, 
constructivism, cooperative learning, and the STS issue investigation and action model (Rubba & 
Wiesenmayer, 1985). The course evaluators asked the instructors to keep a journal, in which they would 
enter their reflections on each of the course developnwnt meetings, and at a later time, provide to the 
evaluators journal entries that they felt comfortable in sharing. 
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The course was organized as a responsibility spiral (Waks, 1992) and grounded in the evaluation 
research of Rubba and Weisenmayer (1985, 1993). Relative to the critical issues of GW, EO, and BC, the 
course was to place emphasis on the underlying science, interrelationships, societal impacts, and public 
policy. Course development included the writing of booklets to serve as the principal course texts. 
Student outcomes proposed by faculty, relative to the three critical STS issues and their interrelationships, 
were as follows: (a) understandings of science and public policy concepts, technological impacts, and 
resources and methods for issue investigation; (b) skills in scientific and technological literacy for reaching 
informed decisions and taking citizenship action towards issue resolution; and (c) intgrests in learning 
more about STS issues. 

The course was offered over the Spring semester, 1994, and met each week for one two-hour 
lecture and one three-hour laboratory period. The three critical issues (GW, EO, and BC) were scheduled 
as separate science and technology "modules": Each module comprised a sequential three to four week 
segment of study and was taught by a different science faculty member. Between each module, and at the 
beginning and end of the course, were placed one to two week "wrap-around" sessions, taught by the STS 
program director and author of the NSF proposal that funded the project. Collectively, these wrap-around 
sessions were known as a unit on Science, Technology, and Environmental Policy. This unit focused on 
social science and public policy concepts and the interplay of public issues associated with science and 
technology. It also provided fomms that engaged students in critical thinking as it pertained to decision 
making and resolution of the three critical issues, and was intended to explicate interrelationships across 
these issues. 

Mfthodology 

Two science educators from the Department of Curriculum and Instruction at the university were 
the course evaluators. These science educators had professional experience in curriculum development, 
implementation, and evaluation that focused on STS themes, including GW. The evaluation design they 
developed was objective driven; that is, the objectives for STS 251 developed by the course faculty were 
used to focus the evaluation. To facilitate triangulation (Erlandson, Harris, Skipper, & Allen, 1993; 
Marshall & Rossman, 1989) of data, a variety of data sources/evaluation means were proposed: 
standardized open-ended interviews with course faculty and students (Gunstone and White, 1992; Patton, 
1990; Rye, Rubba, & Wiesenmayer, 1994); student constructed concept maps (Edwards & Eraser, 1983; 
Novak & Gowin, 1984); student responses to 16 items from the Views of Science-Technology-Society 
(VOSTS) instrument (Aikenhead & Ryan, 1992; Schoneweg & Rubba, 1993); student input via electronic 
mail (e-mail); access to the course booklets and student exam results; and observer-participant (Merriam, 
1988) observation of class sessions. The STS 251 course objectives and respective evaluation means that 
comprised the evaluation design are outiined in Figure 1. The evaluation means described therein target 



Insert Figure 1 about here 



the students who participated in the evaluation. Additional evaluation means relative to the course 
objectives wcits interviews with participating course faculty and the graduate assistant, as described more 

fully below. 
Parficinants 

The pool ftom which participants were recruited for this evaluation were the STS 251 course 
faculty, including the graduate assistant, and the students taking the course for credit. The number of 
students enrolled in this pilot offering of STS 251 was small. It initially consisted of 12 students (equally 
distributed across gender) who were enrolled for credit; two additional students sat in on the course. 
These students rcpicsented a variety of majors, but were m a cross section of the target student 
population for the course. For example, one student was a University Scholar who previously had taken 
two other STS courses and a non-technical course in coUege physics. Another student was a football 
player who had graduated and was in school awaiting the NFL Draft 

So as to limit interference with the course's curriculum and instruction, and to conform to the 
ethical requirements of research and evaluation involving human subjects, the participation of all students 
and faculty in all aspects of the evaluation was voluntary. Eleven of the 12 students enrolled volunteered 
to participate in the evaluation. Additionally, all five course faculty and the graduate assistant volunteered 
to participate in interviews surrounding their instruction. Informed consent was secured from all 
volunteers and monetary remuneration (five to seven dollars for a concept map and 10 to 15 dollars for 
each interview) was provided to those students who elected to participate, regardless of the quality of their 
participation or the information provided. 
Interviews 

Seven different standardized open-ended interview protocols were utilized by the evaluators over 
the duration of this evaluation to conduct private interviews with subjects. AU interviews with students 
and most of the interviews with cojjrse faculty were audio-taped. The interviewers (who were the two 
course evaluators) took notes during the interviews and subsequently listened to the audio-tapes as needed 
to verify and supplement their notes. Most of the audio-tapes were not transcribed due to budgetary 
constraints. 

The evaluators' experiences witli protocols that guided the conduct of interviews early in the course 
infomed the development of protocols utilized to conduct interviews at subsequent points in the course. 
The seven differe^rc interview protocols were as follows: (a) pre-module interviews with students; (b) 
post-module interviews with students; (c) course-drop interviews with students; (d) end-of-course 
interviews with students; (e) pre-instruction interviews with course faculty (the wrap-around unit 
instructors were interviewed at the beginning of the course and the module instructors were interviewed at 
the beginning of their module); (f) post-module interviews with module instructors; and (g) post-course 
interviews with the wrap-around unit instructors and the graduate assistant. 

As described above, the course contained three modules (GW, EO, and BC). Budgetary and time 
constraints did not allow for the scheduling of pre- and post-module interviews with all student subjects. 



Accordingly, a pool of six students was puiposefully selected from amongst all student subjects to 
represent the diversity (gender, ethnicity, major, and academic standing) of students enrolled in the course. 
Subsequendy, two different students (one female aind one male) from this pool were scheduled for pre- 
and post-instruction interviews surrounding each of the three modules. There was a high degree of 
similarity between the pre- and post-module interview questions posed to students. The pre-module 
interview questions are shown in Table 1. 

Insert Table 1 about here 



The course drop and end-of-course student interviews originally were not a part of the evaluation 
design. The cvaluators had planned to conduct course follow-up interviews during the Fall Semester with 
the six students chosen for the pre- and post-module interviews, chiefly to examine any impact the course 
may have had on students' level of interest in and taking action towards the resolution of STS issues. 
However, as explained in the findings section of this paper, student attrition from the course compromised 
the pre- and post-module interview data collection finom student participants, and led to the need to 
interview student participants who dropped the course and to schedule "end-of-course" interviews with 
student participants who remained enrolled in the course. The questions that comprised the course drop 
interview are discussed in the findings of this paper and those comprising the end-of-course interview for 
the student are shown in Table 2. 



Insert Table 2 about here 



Most of the questions in the end-of-course interviews for the students were very similar to those 
posed in the end-of-course interviews for the wrap-around unit instructors and the graduate assistant. For 
questions that asked students to provide self reports of their pre- and post-course understandings relative 
to course content (Table 2, question 4), corresponding questions were posed to the wrap-around unit 
instructors where they were asked to describe major understandings that they thought students developed 
in the course. For queries to elicit what students believed to be important interrelationships amongst the 
three module topics (Table 2, question 5), the wrap-around unit instructors were asked to describe 
understandings of important interrelationships that they believed student's had actually developed. 
Additionally, these instructors were asked to describe how they would have wanted students to answer the 
testimony simulation question (Table 2, question 6). 

The questions that comprised the pre-instructional interviews (Table 3) for all course instructors 
were aimed at identifying (a) the outcomes instructors desired students to achieve by completing their 
module or the wrap around unit, (b) the concepts in their module or the wrap-around unit that they wanted 
students to understand, and (c) the inten-elationships between these outcomes and tlie course objectives 
and all of the course modules. The post-module interviews (Table 4) for the module instnictors revisited 
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the questions about module outcomes and interrelationships and elicited further changes and 
recommendations instructors had in mind for future teaching of their module and the course, 
rnnrept Mans 

During the developmental stages of the course, faculty had expressed some interest in concept 
mapping and the cvaluators were asked to provide a related workshop. Use of the ccmcept map tool 
(Good, Novak, & Wandersee, 1990) held the potential to facilitate course planning and certain emphases 
of the course: critical thinking, creativity, and developing an understanding of the interrelationships 
amongst science, social science, and public policy concepts. Course faculty utilized concept maps to some 
degree in planning their course nxxiules and in conveying knowledge in their respective course booklets 
(see Figure 2) and class presentations. 

Insert Figure 2 about here 

However, concept mapping activities were not planned or executed as pan of any instructors' course 
assignments or tests. 

Therefore, in order for the evaluators to employ concept mapping as an evaluation tool, enrolled 
students needed instruction to develop mapping skills. Accordingly, the evaluators developed a self- 
instructional packet on concept mapping. The packet embraced an inductive approach, where students 
learned the features and attributes of concept maps while engaged in constructing concept maps on a 
familiar topic: household waste. The packet culminated in students preparing a concept map of the topic 
"concept map" (Novak & Gowan, 1984). This packet was provided on the second day of class to all 
students who volunteered for the course evaluation. 

Accompanying the concept map self-instructional packet was the first of four concept map 
"assignments" to be completed voluntarily as a pan of the course evaluation. This assignment instructed 
students to map several concepts central to the first module and that cut across the entire course. Most of 
the concepts to be mapped were provided to students by the evaluators, who had gleaned them from pre- 
instructional interviews (Table 3, question 4) with the instructors of the first module and the wrap-around 
unit. This assignment, shown in Figure 3, was intended to elicit students' understandings of these 
concepts at the beginning of Uie course; die additional concept mapping assignments were to explicate 
students' understandings at the end of the first module and surrounding die other modules and at the end 
of die course. The instructors of the course modules agreed to prepare a concept map of the concepts 
provided in the student concept map assignments, afisr they completed the instruction of their nxxlules. 
These instructor-prepared concept maps, which were to set forth the understanding that each instructor 
hoped enrolled students had developed as a result of instruction to date in the course, were to be used by 
the evaluators in assessing the student prepared maps. 



Insert Figure 3 about here 
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As shown in Table 1 (questions 5 and 6), the student-prepared concept maps were also to be 
utilized as an auxiliary visual in the student interviews. The maps were to serve as a basis for probing 
students' understandings about concept relationships explicated in their maps. 
Class Session Observations 

As set forth, the evaluation design (Figure 1) did not include class observations as a majsa: data 
source. However, these observations did contribute in important ways to the course evaluation, as 
discussed below. Class sessions included lectures, forums, and laboratories. Collectively, the two course 
evaluators attended the majority (21) of these sessions, however, the evaluators did not stay for the 
duration of the laboratory periods. The evaluators kept each other informed, via electronic mail and 
conferences, on their class observations. 

The principal reason for the evaluators attendance of class sessions was to keep abreast of "what 
was happening" in class. This included taking notes on class attendance, course content conveyed, 

teaching methodologies employed, and student-instructor interactions. This immersion of the evaluators in 

the course class sessions was /iewed by the evaluators as important to the conduct and interpretation of the 

interviews with students and instructors. 

To limit their influence on course instruction, the course evaluators did not pose or answer 

questions, or comment otherwise, during class sessions. However, the students and faculty knew the 

evaluators' identities: Their presence during class sessions was obvious, and this presence may have had 

impact on instruction. 

FlPPtrnnic Mail 

Electronic mail (e-mail) was a planned component of STS 251. Students were provided with a 
"hands-on" orientation and user account for the duration of the course. Students and faculty coi-M post 
messafe. ■ for other STS 25 1 students and faculty to read, and respond to as desired, on a course bulletin 
board; students and faculty also could correspond privately with any other students, faculty, and the 
course evaluators. Course evaluators monitored the e-mail postings. They also soUcited, via 
announcements in class, private e-mail messages from students participating in the evaluation on how the 
course was going (e.g., likes and recommended changes). 
vnSTS Instrument 

Data were collected on students' views about STS interactions at the beginning and end of the 
course using 16 multiple-choice items selected from the 1 14 items in the Views on Science-Technology- 
Society (VOSTS) item pool, Form CDN.mc.5 (Aikenhead & Ryan, 1992). These were the same 16 
VOSTS items used by Schoneweg and Rubba (1993; Bradford-Schoneweg, Rubba & Harkness, in press) 
to assess students' views about STS interactions in the preceding course, STS 200-Critical Issues in 
Science.Technology and Society. From that work, it was assumed that the VOSTS items possessed 
vaUdity and were reliable for the purposes of this assessment The data analysis procedures foUowed are 
described by Bradford-Schoneweg, Rubba and Harkness (in press). 

In summary, this evaluation design, as conceived and modified as needed by the evaluators, 
represented their efforts to carry out a "practical evaluation" (Patton, 1980) that involved stakeholders and 



was creative yet situationally responsive. The qualitative emphasis of this evaluation provided for an in- 
depth examination of multiple and unique perspectives of the course faculty and enrolled students. The 
prolonged engagement that the evaluators had with the course, and the awareness they maintained of the 
course content and dynamics from observing class sessions, were considered very important to the 
trustworthiness (Erlandson, Harris, Skipper, & Allen, 1993; Patton, 1990) of the assertions and 
recommendations that emerged from the findings of this evaluation. 

Findines 

Over the period of the semester, two students who had volunteered for the evaluation dropped the 
course after completing a portion of the course. Given the small size of the course, these drops had a 
greater than normal impact on the availability of student participants for the interviews. Interview data 
were not available for some modules from the same students pre- and post-instruction. Additionally, two 
other factors compromised carrying out the evaluation dcsi\?\ as conceived: (a) Few students elected to 
complete the concept map course evaluation "assignments'" jt the beginning of the course (Figure 3) and at 
end of the first (GW) course module; and (b) Most students elected not to provide input to the evaluators 
on the course via e-mail, despite a number of requests made by the evaluators to do so. As a result of the 
above, several measures were taken. Two student subjects who dropped the course agreed to and 
followed through on their commitment to a "course-drop" interview, which queried them on reasons for 
dropping the course as well as perceived strengths of and changes needed in the course. An attempt was 
made to interview at the end of the course all student subjects who remained in the course: Six of the 
students agreed to an interview and four followed through with their commitment. The concept maps (and 
corresponding interview questions. Table 1, items 5-6), e-mail, and the course follow-up student 
interviews were abandoned as data sou-xes. Additionally, the reflective journals that the evaluators asked 
the course instructors to keep during course development were not u^ed as data sources by the course 
evaluators. 

The major sources of data for this evaluation came from the interviews conducted with students, 
the module instructors, and the wrap-around unit instructors and the graduate assistant. Additional 
information was gleaned from the informal class observations, the course booklets, and the VOSTS item 
responses. Findings from each of these sources are summarized below. 
Tnferview «i with Students 

The course evaluators were able to obtain a prc-module interview on five students, but only able to 
obtain ixilh prc-module and post-module interview data on three students. Still, the experience of 
conducting, and data obtained from, these interviews were very useful to the course evaluators in 
formulating inteiview questions to comprise the end-of-course (Table 2) student interview protocol. Data 
from these end-of-course interviews was obtained fix)m four of the six students who had agreed to this 
interview. The evaluators do not make the claim that these four students were representatives of all 
students enrolled in the course. However, these four students did represent the gender distribution of the 
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students who remained enrolled in the course and included students who matched the target audience for a 
200 level science course for non-science majors. 

Some of the questions that comprised the pre-module, post-module, and "course drop" interviews 
were aimed at collecting or revealed data on some of the parameters targeted by the "end-of-course" 
interviews. Two of the six students who had agreed to but did not follow through with an end-of-course 
interview did complete a pre-module or post-module interview during the last month of the course. 
Additionally, the evaluators did have interview data from two other students who had dropped the course 
after completing a portion of the course. These four students' responses, where appropriate, were pooled 
with data obtained from students who completed the end-of-course interview. This pooled data, which 
represents the reponses of from four (33%) to eight (67%) of the students who enrolled for credit in the 
course, constitutes the principal findings of the student interviews and is described as such in the section 
that follows. Subsequently, additional data from the pre-module interviews is presented that regards 
students' reasons for taking and outcomes they desired to obtain from the course. 
Principal Findings 

Most of the parameters on which the interview questions elicited student reponses are presented 
below. The total number of students (e.g., four of seven) who comprise the data-base for the findings on 
any parameter is noted when this number exceeds those four students who completed the "end-of-course" 
interview. 

Major Strengths of Course. All of the students (six) stated directly or inferred that the course 
instructors as individuals were a major strength. In connection with this, various students perceived the 
instructors to have certain "positive" characteristics, e.g., flexible, receptive, very intelligent/experts, and 
to care about teaching. One of the students had this to say about the instructors: 

You know, one of the strong points of the class was the fact that all the teachers seemed to pretty 

much care what they were doing [Y]ou'd see a lot of the professors of the other modules 

sitting in on the class, even if it wasn't their module and to me that was a sign they were, you 
know, interested. 

Two students cited "small class size" and the course topics and various rcsourcfes as strengths. 

Changes Recommended. The question (Table 2, question 2) to elicit students' suggestions for 
changes in the course did not ask students about any scecific aspects of the course, i.e., it did not "direct" 
their thinking towards any given course parameter. Therefore, student responses likely reflected what was 
most pressing on their mind in terms of modifications they would like to see made in the course. Four of 
the eight students inferred that it would be good to get students more involved. One student described this 
as a need for the course to be more "thought provoking." Another student suggested requiring students lo 
keep a journal in response to thought provoking/reflective questions and hand it in periodically. Three 
students talked about the need for the course to be more organized; two of them referred to the course as 
being too "scattered." Three students also suggested that the EO module was too technical or there was a 
need to reduce some of the concepts therein. Another suggestion, mentioned by two students and that 
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surfaced again later in these findings, was that both faculty and students needed to be more active on e- 
mail. 

Course Scheduling. Although two (of eight) students did state that they liked the schedule as 
is, thi«e others stated that the two-day, two and three hour time blocks over which the course was 
scheduled made it difficult to schedule other courses. Three students also voiced the need to have class 
meet three times per week. One of the students put it this way: 

I keep on feeling that this time is being wasted. I'm not quite learning as much as I could. It 
probably has to do with the fact that it's [class] on Mondays and Fridays. So, a whole half of a 
week goes by and I don't retain. 

Number of Instructors. Most (four of six) students favored having at least three instructors 
for the course. Reasons cited included the array of expertise and the different viewpoints the instructors 
presented. However, students did bring up the concern that it was hard to adjust to the changes in 
professors and that they needed to coordinate moreso: 

Lecture. The most frequent comments made (by three to four of six students) were that the 
lectures were either good or interesting or that they were reiterations, or at least represented the majority, 
of what was present in the readings. 

Readings. There was no trend present in the comments received about the course readings. 
Some students found the need to have more readings and to have the booklets standardized across the 
modules. Other students found desirable the technique employed in one booklet of "boxing in" certain 
information, or that the modules presented information that was worthwhile keeping as a reference. 

Forums. Two students saw these to be the "most valuable" part of the course. A response from 
one of these students follows: 

[T]hey gave out questions, they split us up in groups and it was really cool because, for example, 
the EO module— I was in charge of the big business part of it and it was really fun because even 
though I was against it, we were the people . . . who were putting all the crap in the air . . . all wc 
wanted to make was money. We didn't care about the environment. [E]ven though I completely 
disagree with that, I had to be placed in that situation and I had to think of what the big businessscs 
are saying to environmental groups and to the government. 

A third of the students mentioned elsewhere in the interview that the forums were a very positive 
aspect. Attributes of the forums cited by students included the open discussions, the instructor feedback, 
and the focused questions provided. 

Laboratory and Associated Reports. Most frequently (five of six students) comments were 
made that pertained to the need to make some modifications sun-ounding the GW laboratories, e.g., two 
students remarked that the quantity of work was too much "all at once" and one student said more needed 
to be done with computers. Three smdents remarked that the EO and BC laboratories were liked a lot/well 
put together. One student remarked that the lab enhanced his/htr writing skills. 



Amount of Work. The majority (four of seven) of students commented that the course was 
either a lot of woric, too much work, or more woric than the amount expected. Yet, three students stated 
the quantity of work was not excessive. 

E-mail. There was no distinct trend in the responses student provided. Two (of eight) students 
remariced that they loved it or that it was the favorite part of the course. However, other students were 
critical of various aspects surrounding e-mail use in the course. Two of these students inferred that having 
e-mail competence as a course prerequisite, or helping students get "up to speed" more quickly, would 
have been advantageous. Two other concerns voiced by individual students were that the faculty were not 
responding enough, and that it should be mandatory for students to post one mesage on a certain day each 
week. Yet another student had this to say about e-mail: 

I mean it's kinda neat to be able to sit down and write someone ... and see their viewpoint, but 

we should be able to do that as a class in person If we can't do that in from of each other 

ther's something wrong that's part of our problems as a society, I think. 

Absences. Three (of six) students inferred that student absenteeism, in part, may have been due 
to the fact that the lectures were reiterations of or pretty much the same as the readings. Other reasons 
attributed to class absenteeism were that attendance was not expected, in part due to the course being a 
pilot offering: 

I think that a lot of people just thought this was a pilot course land] it didn't mean anything 

{That] they can get away with a lot of things. 

Course Prerequisites. There was no trend apparent in students' responses. One (of five) 
students believed a basic physics course should have been listed as a prerequisite; another, that e-mail 
"literacy" should be a prerequisite. 

Natural Science General Studies Nature of Course. All students stated that this course 
should count towards fulfilling the undergraduate general studies "natural science" credit requirements. 
Most students gave as a rationale that this course presented a diversity of and interrelationships between 
science disciplines and topics. 

Students' Perceptions of Own Understandings. Students cited several ways in which this 
course added to their prior knowledge. The understanding cited by most (four of six) students was that 
they now knew the mechanisms that accounted for the greenhouse effect or how GW "really works." In 
connection with the latter, one student remarked: 

Now I can now talk with people on a pretty intelligent level. 

The second most frequent (three students) knowledge gain described was an understanding of energy 
efficiency. Two students stated their understandings had increased relative to nuclear power, the 
greenhouse gases, and how power plants worked. 



Interrelationships Amongst Course Topics. The most frequently cited (five of six 
students) interrelationship was that our energy consumption effects GW and/or biodiversity. Additionally, 
three students remarked that GW, EO and BC do not stand on their own: They are interconnected. 

Testimony Scenario. The four students who participated in the "end-of-course" interviews 
were asked to choose one of three testimony topics, "walk" the interviewer through the preparation and 
delivery of this testimony, and answer other related questions (see Table 2, questions 6-8 ). In addition, 
the student who completed the post-nxxiule interview near the end of the course was placed in a similar 
scenario: The exception was that the post-module interview specified, as opposed to allowing smdents to 
choose, the topic of the testimony. Thus, the findings reported below represent five students. 

It is important to acknowledge that this course did not attempt to teach students how to prepare and 
deliver a testimony. Thus, the purpose of this "testimony scenario" was not to assess students' abihties to 
prepare and dehvcr a testimony. Rather, this scenario provided an appHed setting through which the 
evaluators could examine smdent's abilities relative to certain course objectives (Figure 2, items Id, 2b, 
and 2c). In a broad sense, this scenario provided data relevant to the assessment of all of the "knowledge 
and understanding" course objectives. The results from these five "testimony scenarios" are presented and 
discussed below. 

AU but one of the students selected "EO for the future" as their testimony topic. The reasons 
provided by smdents for making this selection included the following: 

It is one of the most feasible areas to make changes in at the personal level. 
I think it affects the other two issues. 

The root of the problem seems to be our energy uses-that's what's driving thoses two things [GW 
and problems associated with BC]. 

As a means of preparing for their testimony, all students indicated that they would do library 
research. Most students indicated that they would (a) speak to professorsor other experts in the field as 
another means of researching their topic, (b) contact governmental centers or utilize government resources 
that deal with environmental issues, and (c) survey the public to ascertain pubUc opinion. Additionally, 
most students identified specific resources or knowledge from STS 251 that they would utilize in 
testimony preparation, e.g., the GW emissions lab, the course readings, the professors, alternative fuels 
research, and the video tape After the Warming (Ambrose Video Publishing). 

When asked what they would do with a resource that they discovered during their preparation that 
ran contrary to their position on the issue, most students said they would "present it" as part of their 
testimony so as to represent botii sides of tiie issue. Comments made by these students surrounding the 
use of "tiie opposite view" included tiie following: 

And we learned in psychology that a well rounded discussion tiiat gives all sides of a debate- 
intelligent people are a lot more receptive to tiiat type of a presentauon. People arc a lot more 
distrusting of a presentation tiiat is all one sided. 
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Well, I would read it, see if I could find some sort of other information about it-see what's going 
on here. Then [I would] look at how my ideas are beter, if I thought they were better, and figure 
out ways of addressing that opposite viewpoint. 



The major points students said they would make in their testimony revealed a variety of concepts 
inherent in STS 251, however, students tended to make different as opposed to similar points. The only 
distinct point made by more than one student was that they would focus on the hidden/environmental costs 
of EO and energy consumption, e.g., traditional energy sources look cheaper now, but if they reflected 
"real" costs, alternative energy sources would be competitive. Other concepts and concept 
interrelationships mentioned by individual students included energy efficiency, energy conservation, the 
negative effects of GW on biodiversity, and the potential of the solar energy option to combat GW. 

AU but one student stated they would recommend actions be taken towards the resolution of their 
testimony issue. The only action stated by a majority of the students was the need to practice energy 
conservation. Two students stated "recycling." Two students also desnibed political involvement, e.g., 
write legislator. 

Three of the students said their preparation for the testimony would have been the same or similar 
regardless of whether or not they had taken STS 251. However, one of these individuals did say he/she 
would have been less likely to survey the public prior to this course and another remarked that this course 
would enable him/her to prepare a testimony more quickly. One of the students said preparation of his/her 
testimony would be very different as a result of what was learned in the course, due principally to a much 
greater understanding gained of the issues that comprised the course. 

In order to validate students' perceptions of the effect STS 251 had on how they would prepare a 
testimony, pre-course data from these students on testimony preparation was necessary. The course 
evaluators lacked such data, however, three of the students did participate in pre-module interviews early 
in the course and also in subsequent (post-module and/or end-of-course) interviews, all of which placed 
these students in this testimony scenario. A study of such pre-module and subsequent interview data 
regarding student preparation of testimonies suggested that students did not change appreciably the way 
they would go about preparing for the testimony, thereby lending support to the students' perceptions. 
For one student no changes were obvious, but the other two students did make additions that reflected 
course content and resources, e.g., using the professors in the course as a resource and researching 
greenhouse gases. 

A much greater difference between the pre-module and subsequent interviews was obvious in 
terms of the "key points" students would make in their testimony and the actions they would recommend 
towards issue resolution. In terms of the key points, students tended to be more organized and grounded 
in a science base: The points reflected a substantial amount and variety of course content and generally an 
accurate understanding of that content. Such key points in one testimony on the threat of GW included 
citing the potential consequences of GW, in an effort to get the audiences' attention, and explanations of 
the feedback processes related to such consequences, e.g., deforestation leads to a decreased sink for 
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atmospheric carbon dioxide. Key points made in the subsequent interview testimonies on EO for the 
future included the separation and definition of EO as fossil, nuclear, renewable, and a focus on energy 
efficiency and conservation. Actions for issue resolution included supporting different environmental 
organizations, simple (physical) things folks can do in their everyday lives that don't require a lot of effort, 
and policy formulation at the community level. 

Interest in STS/Related Courses. Students reported being more interested in STS now and 
attributed this to being more aware of ways people affect and can help the environment. Two students 
specified they would be interested in taking future courses along the Unes of BC and EO. One student 
remarked that he/she had been thinking tiiat STS would be a great major, but cited "abUity to do the 
science" as a barrier and acknowledged that he/she could be part of "the movement" regardless. 

Resolution of STS Issues/Lifestyle. Three students reponed that the course influenced tiieir 
thinking about the resolution of STS issues. Examples of explanations given by tiiese students foUows: 

First, I didn't realize tiiat tiie problem was this serious. I think they can be resolved. . . .1 didn't 
think that was possible before the course.. 

Before I thought tiiat global wanning was sometiiing people agreed upon. . . .1 now realize how 
difficult it wUl be to resolve tiiese issues because people do not realize how complicated tiiese 
issues are. 

One student said the course generally had littie impact on his tiiinking about issue resolution 
because it dealt little with tiie philosophical aspects, e.g., how we need to come up with new ways to view 

ourselves as a part of nature. 

AU tiie students reported tiie course influenced tiieir thinking about tiieir own Ufestyle and provided 
several examples. All students spoke to their own use of energy (e.g., being more cognizant of how much 
energy they use) and related efforts to conserve energy (e.g., drive less and buy a bike). Two students 
discussed their interest in joining an environmental organization. One student remarked tiiat he/she now 
watches and reads more environmental shows/articles because of tfie knowledge and interests gained 
throughout the course. 

What Else Evaluators Should Know. The questions that led up to this last item encouraged 
students to reflect on and talk about many course aspects, and tiiereby informed in a broad sense tiieir 
tiiinking about "what else is imjMianl for tiie evaluators of tiiis course to know." Accordingly, we believe 
the student responses to tiiis last item are very credible and the positive nature of tiiese responses speaks 
highly of the pilot offering of tiiis course. 

Six of tiie eight students specified tiiat the course was eitiier very inforaiative or worthwhile. 
Individual students reported "learning a lot," that tiie course presents knowledge that students "should . 
have," tiiat tiic forums were a very positive aspect, and that what was learned would "stick with" the 
student longer because it was related to real worid problems. One student had this to say: 



ERIC 



[LJet's face it: Most students who are taking "Geology they arc not dealing with that once 

they arc done with class, you know. They don't understand-there is no sense of importance of 
what they've leamed-whereas at least with this class it does certainly give you that. I don't see 
how it couldn't, you know? 

Five students gave suggestions for various improvements that should be made in the course. Most 
fiequently the need was cited for better course organization and/or better communication with the students. 
One student suggested there was insufficient emphasis on making the course content "thought provoking" 
and another student suggested that the quality of the infOTmaticMi could be a bit nwre rigorous. Two 
suggestions were made that could potentially impaa ftiture enrollment in the course: (a) Use the word 
"environment" in advertising the course, as many students are already interested in helping the 
environment; and (b) The course may be of interest f ludents in the agricultural sciences, as each issue 
presented is agriculture related. 

Sfiiflpnt Interests and P psirpd Oiifpomcs 

The prc-module and course drop interviews attempted to disclose students' reasons for taking STS 
251 and what they hoped to get out of the course. A total of six diftail students completed these 
interviews. Student responses were seen as potentially useful to the identification of salient features for 
future mariceting of the course. All but one of die students mentioned the environment (e.g., interest 
in/concern) in articulating reasons for taking STS 251. Most of the students also mentioned, as a reason 
for taking the course, that they perceived a relationship between the content of tiiis course and a college 
major (undergraduate or anticipated post-graduate) that emphasized the environment or ecology. One-third 
of the students mentioned that they liked the idea of several professors teaching the course and the 
interactive or interdisciplinary nature of the course. 

The majority of the students identified as a desirable course outcome an increased or better 
understanding of the course subject matter and environmental problems. Two students identified each of 
tiie following as desirable outcomes: (a) Knowledge of the details of what goes on "inside" or beneath the 
surface— how things really happen; (b) The abUity to pass course knowledge on/get otiiers to understand 
the problem; and (c) Learn things that I can do to help die environment. 

Individual students talked about wanting to increase their ability to think critically and look at 
issues from all sides as well as how to relate the content of tiiis course to the present In connection with 
the latter, one student described as a desirable feature: 

It made me think more about the worid than [text] books. 

lntfrview« i with Tnstnictors 

These interviews consisted of pre-instruction interviews with all course faculty, post-module 
interviews witii the module instructors and post-course interviews with the wrap-around unit instructors 
and the graduate assistant 



Pro-Tiistru rf^nn Interviews 

The pre-instruction interview protocol (Table 3) targeted desired outcomes for students, concepts 
central to instruction, and interrelationships aaoss the entire course. A summary of the three module and 
the two wrap-around unit instructors' responses are presented below. 

Insert Table 3 

Student Outcomes Desired and Their Importance. In response to question 1 (Table 3), 
each of the three module instructors provided between five and sbi outcomes that he desired students to 
achieve as a result of completing his module. The majority of outcomes appeared to be at the knowledge 
and comprehension levels of Bloom's Taxonomy (Hassard, 1992), and are set forth in Figure 4. The 
category headings generally reflect the labels used by the instructors in their actual description of each 
outcome. 

There appeared to be Uttle overlap across the three modules in terms of the specific outcomes 
desired by each instructor, i.e., any outcome articulated by one instructor generally was not mentioned by 
the other two instructors. A discussion of the importance of these outcomes (Table 3, question 2) with the 
module instructors led to a disclosure of additional student outcomes that the instructors deemed imponant 
as well as some interrelationships of these outcomes across the science modules and with society and 
technology. The following exempUfy the previous and stand out amongst the instructor responses: (a) 
Students need to come to a common understanding of the physics behind greenhouse warming; 
(b) Students need to understand that social and economic forces, such as consumption of fossil fuels, 
drive greenhouse warming and the achievement of energy independence by the United States wUl not solve 
greenhouse warming; (c) Students need to know the reasons— political, technological, and societal— why 
the soft energy path is often not chosen as well as the real cost [of energy], which includes the 
environmental impact of energy consumption; (d) BC presents a gate keeper approach to ecology in that it 
is [more] concerned with individual species within the ecosystem as opposed to the dynamics of the entire 
system; and (e) Students need to be able to participate in poUtical discussions on BC and understand 

regulatory issues related to BC. 

The two wrap-around unit instmctors expressed similar views concerning desired student 
outcomes and the importance of these outcomes. Both instructors expressed the need for the course to be 
intellectually exciting for the students and for themselves. This was seen as important for the students in 
order to stimulate involvement in the course during class meetings, and long term retention and application 
of the content in life. Each instructor also indicated that they personally desired inteUectual stimulation and 

excitement when teaching. 

Both instructors emphasized the need for the course to emphasize higher-order goals. For 
example, one instructor voiced the belief that coUege students are already environmentaUy aware but feel 
disempowered— students knew issues existed, but did not know what they could do to help resolve them. 
He saw a need for the course to help students appreciate the complexity of policy formation, with respect 
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to each of the three STS issues, as a way of empowering them to act. The other instructor noted, in 
particular, that he wanted students to understand (a) how the various institutions that make decisions in our 
society work, (b) that there are a variety of ways for individuals to enter into those decision making 
processes, and (c) because these are very complex issues, the issues are not easily resolved and the 
outcomes can be equally complex. 

One wrap-around unit instruaor also indicated that the course and the anticipated outcomes were 
crucially important to preparing citizens for decision making. Complementary to this, the other instructors 
noted that GW, EO, and BC were not the problems university undergraduates of a hundred years ago 
needed to be able to face. But, as society has faced new problems, the charter of the university must 
change as it has in the past to help graduates meet the extant demands of citizenship. As a result, he 
indicated, these goals have legitimacy in general studies science courses like STS 251. 

Outcomes and Course Objectives. The three module instructors were asked (Table 3, 
question 3) to identify the relationships between their desired student outcomes and the course objectives 
(Figure 1) for the entire course. All three instructors specified that the student outcomes desired would 
facilitate smdent knowledge and understandings of GW, EO, and BC (Figure 1 , objective la) and skills 
and interest in reading more about and taking additional course work in these three areas (Figure 1, 
objectives 2a, 3a, and 3b). The following were instances of relationships between "student outcomes 
desired " and course objectives mentioned by just one instructor: (a) Investigate a problem related to GW 
(Figure 1, objective Id); (b) Develop skills in accessing and retrieving library information on BC (Figure 
1, objective 2b); and (c) Promote interest in taking more STS courses (Figure 1, objective 3b). 

Both of the wrap-around unit instructors saw close ties between the outcomes they noted and the 
course objectives. Both instructors saw the two goals they noted as conesponding to the knowledge and 
skills sections of the STS 251 objectives, and pointed to particular correlations. 

Central and Interrelated Concepts. As part of the evaluation design for this course, the 
evaluators intended to examine concept maps constructed by the smdents of concepts identified by the 
module instructors to be "central to each module" and "interrelated across the modules." These concepts, 
as well as concept maps, were also intended to form the basis of some of the pre-module and post-module 
interview questions with students. Therefore, each instructor was asked (Table 3, question 4a) to identify 
the most central/important concepts for his module. These concepts are set forth in Figure 5. One module 
instructor also provided a concept map (Figure 2), illustrating a greater number of important concepts than 
is reflected in Figure 5 for his nxxlule. 

Furthermore, each instructor was asked (Table 3, question 4b) to identify, amongst these big 
ideas/concepts, the three concepts that were most interrelated with at least one or both of the other 
modules. Each instructor identified three concepts, resulting collectively in the following nine concepts: 
fossil fuel consumption, agricultural production, land use changes, efficiency, exponential growth, 
pollution, carbon dioxide cycle, energy flow, and primary productivity. 
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As explained previously, the concept mapping evaluation component was not implemented 
successfully. Still, the concepts identified by the instructors may inform course revision efforts to 
interrelate further the three modules. 

The wrap-around unit instructors saw the central concepts in their unit also as being big ideas in the 
entire course. These concepts/big ideas included the foUowing: pubUc/coUective good, the pubUc poUcy 
formation process, and the involvement of governmental environmental agencies, Non-Governmental 
Organizations, and international meetings and agreements (e.g.. United Nations Conference on 
Environment and Development and the BioDiversity Treaty). 

Interrelationships Across Course. The module instructors were asked (Table 3, question 5) 
to identify interrelationships across the modules and the wrap-around unit that comprised the course. The 
interrelationships identified were as foUows: (a) Energy use plays a large role in global climate change, 
e.g., fossil fuel combustion leads to carbon dioxide accumulation, thereby enhancing GW; (b) The 
environmental impact of the generation of energy threatens biodiversity; and (c) The environmental 
problems addressed share base causes. 

AU of the module instmctors spoke to the need for the poUcy wrap-around unit to help puU together 
and make visible the interrelationships between the three modules. Indeed, one instructor specified that the 
surfacing of significant interconnections among the modules would depend on whether or not the wrap- 
around unit "built the bridges." One module instructor spoke to the need for the instructors of the 
modules, as well as the wrap-around unit, to bring out interrelationships with and implications for society. 

Another module instructor voiced concern that faculty were working too independently and 
expressed a desire to integrate the modules into a single unit The latter was seen as a way to enhance 
greatly the coonlination of GW, EO, and BC and preserve the logical progression of the presentation of 
science concepts, as opposed to that progression inherent in teaching separately three interrelated modules. 
An example was provided: The concept of fossil fuels and related utilization should be developed fuUy 
during the presentation of GW, not several weeks later. 

One of the wrap-around unit instructors compared the wrap-around unit to the glue that held the 
three modules together. The other instmctor referred to this unit as the comfortable window into the 
course for those students who are shy with science. He saw two other connections: (a) He expected the 
unit to handle the intersections needed between the three modules that the module professors would not 
handle or could not take time for— that is the focus on social decision making related to the issue; and (b) 
He expected the wrap-around unit to set-up the learning context for each unit and to Unk it to the previous 
one. Both unit instructors saw the need for them to understand tiie other nwdules and their 
inteirelationships. They also saw the potential for a lot of "snafoos" in the course given the number of 
instructors involved. 

Other Important Aspects. In response to question 6 (Table 3), two other course related 
aspects surfaced that the module instructors felt the course evaluators should be appraised of. Two of 
these instructors brought up concept mapping, which was a topic in pre-course workshops for the 
instructors and was a planned (but unsuccessful) component of the evaluation design set forth by the 
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evaluators. One instructor commented on his exposure to concept mapping and specified that it helped him 
to identify, through the linkage process inherent in concept map construction, the central concepts in his 
module. While making reference to the concept miap (Figure 2) he had prepared as an organizer for his 
module, he had this to say: 

Instructor: And I found this pretty useful, this idea of concept maps. I don't know about— [you 
will] still have to see what happens if you teach with it or try to evaluate it But just 
as a way of outlining the course, I thought it was very useful. 

Interviewer So you found it useful as an organizer for yourself? 

Instructor: Yes. Because it forced you to try to come up with the important outcomes here, as 
opposed to just— because they had to link together in this way. Then things you felt 
you would have had fun talking about but didn't fit nicely into this, you suddenly 

realized weren't as important as So I thought it was useful in that way— it lets 

you concentrate on what you really need to get over in a short period of time. 

Another module instructor said that he had planned to prepare concept maps of his lecture topics 
and provided a suggestion: If the evaluators were going to employ concept mapping as one means of 
collecting data for course evaluation, they need to have some way to assess student competence in the 
concept map construction process. 

The other topic that surfaced had to do with one module instructor's dissatisfaction with straight 
lecture as a mode of teaching. He planned to explore with students the everyday definitions and ideas they 
had for various concepts in his nxxiule and believed such explorations may facilitate learning the scientific 
definitions for such concepts. He also wanted students to come to realize how scientists conceive of and 
approach problem solving and believed such awareness would help the students relate better to the science 
in his module. 

Post-Module Interviews 

All three module instructors completed an interview (Table 4) subsequent to teaching their module. 



Insert Table 4 about here 



As indicated by a comparison of the pre-instruction (Table 3) and post-instruction interview questions, 
instructors were asked to provide their perceptions on some of the same parameters before and after 
instruction. Although not reported below nor indicated by the post-module interview questions (Table 4), 
two of the three instructors also were asked to and did submit concept maps of specific concepts, which 
refiectcd an understanding that each instructor hoped students had developed up to that point in time in the 
course. A summary of the module instructors' responses relative to the parameters investigated follows. 

Student Outcomes Desired. Instructors were queried (Table 4, question 1 ) as to what, if any. 
changes they would make in the student outcomes desired, as identified during their prc-module interview 
(and reported above in this paper). Furthermore, the instructors' perceptions of the degree to which these 
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outcomes were achieved was obtained. None of the instructors deleted or changed any of the outcomes. 
However, one instructor specified that he should have emphasized to a greater extent the social context of 
EO. 

Collectively, the instructors perceived that the majority of these outcomes were achieved. Time 
was reported to be a limiting factor by all instruaors in regards to enabling students to achieve, to the 
extent desired by individual instructors, these outcomes: (a) the role of social factors in driving as well as 
constraining solutions to greenhouse warming; (b) an understanding of alternative options for future 
energy policy; and (c) evolution. One instructor also discussed two additional outcomes that students did 
not achieve to the degree desired: (a) Smdents perceptions of global cUmate change as a problem did not 
appear to have changed; and (b) Students appeared to not be able to communicate to others very well the 
problem of global climate change. 

Evidence that instructors cited to support their perceptions of smdent achievement of the desired 
outcomes included e-mail postings by students, class participation, course examinations, and course lab 
reports. Problems were cited in regards to some students' abUity to write in an organized and effective 
manner. 

Two of the instructors reported (in response to question 2) that they would not change the Ust of 
student outcomes desired if they were to teach the module again. One instructor inferred that he would add 
an outcome related to the utiUzation of space, or more specifically, land use. The latter was seen by this 
instructor as a unifying concept in regards to biological and socio-political aspects of biodiversity. 

Changes in Modules. All three instructors identified, more broadly, changes they would make 
in their segment (module) of the course for future instruction. These changes were in the areas of 
laboratory resources and assignments, as well as course lecture and written materials. In regards to the 
laboratory, changes recommended included procurement of computer software on (a) emissions relative to 
energy consumption and GW and (b) land use (e.g., a program that contains variables such as population 
density and type of human diet and calculates the required land area and fossil fuel inputs). One instructor 
specified he would increase the time spent on the external costs of energy consumption, such as the 
environmental impact of mining and the effects of air pollution on human and animal health. Another 
instructor said he would give students a more extensive pretest in order to determine their prior knowledge 
in the chemistry and biological aspects important to an understanding of his module; and that he would like 
to work more so on building instructor-student rapport. 

One instnictor also made suggestions pertinent to the enth-e course: (a) Three 50-minute lecture 
periods and one 2-hour lab period should comprise the course schedule; (b) The distinction between the 
lecture and laboratory components should become more obvious to the student; and (c) The character of 
industrial society should be given more attention in the course. 

Interrelationships Across Course. Although instnictors were asked about this during their 
pre-instniction interview, this was posed aable 4, question 3) again to note any changes in perceptions 
about course interrelationships. This question revealed little new information. Two of the three 
instnictors specified again the need for the poUcy wrap-around unit to pUll together the three nxxiules; to 



make visible the interconnections between them. One instructor felt he needed to read the other two 
modules and the wrap-around unit more extensively before commenting. This instmctor specified that the 
science and technology modules were interrelated in that they were each instances of how industrial 
changes impact the environment; fossil fuel consumption and land use were identified as concepts tiiat cut 
across these modules. 

Recommendations to All Faculty. Because these post-module interviews occurred part way 
through as opposed to at tiie end of the course for two of tiie three instructors, the recommendations they 
made in response to question 4 (Table 4) do not collectively represent views based on the entire duration of 
the course. Nevertheless, this data enriched tiie pool of suggestions tiiat can inform course revision. 
Responses to question 5 (Table 4) have been collapsed witii tiie question 4 data, as tiiey also provided 
recommendations towaids course improvement These recommendations, summarized in outiine form 
below, provided specific suggestions for the policy wrap-around unit, issues forum, first day of class, 
harmonization of tiie modules and assignments, quantity of course content, e-mail, number of course 
instructors, and student evaluation. 

(1) The policy wrap-around unit should bring out, more so, tiie interconnections among tiie tiiree 
modules. The latter might be facilitated if tiie wrap-around unit devoted more time to Agenda 
21 (United Nations Conference on Environment and Development, 1992) 

(2) The issues forums provided a positive atmosphere in which students voiced tiieir opinions and 
views. Accordingly, this component of tiie course should be strengthened. One idea is to 
establish groups at the first forum and students would remain in this group for tiie remainder of 
tiie course. To each group, a topic of investigation would be assigned at tiie beginning of each 
module. Group members could be assigned specific roles (so as to equalize tiie workload) and 
meet at various points throughout tiie course witii instructors and/or tiie graduate assistant. A 
genuine interest must be shown by the instructors in each group's activities. 

(3) The showing of the 'ideo tape should be omitted from tiie first day of class. This is a 
"passive" experience and one similar to tiiat which students have experienced before. Instead, 
considerable time should be spent articulating tiie rationale for why GW, EO, and BC were 
chosen as tiie critical issues, as opposed to any tiiree science and technology topics, to 
comprise tiie bulk of this course. Furtiiermore, tiie course topics in the syllabus should be 
examined and ways identified in which tiie tiiree science and technology topics are ■ 
interconnected. 

(4) Students need to be very clear, fi-om the start, on tiie course basics, e.g., testing, assignments,, 
etc. The text of tiie modules and course assignments need to be harmonized. One evaluation 
component was described tiiat, if added, may enhance student interest in tiie course: Students 
could declare an optional personal objective tiiat would specify an outcome which could be 
graded. This would comprise a portion of tiieir course grade and may take tiie place of one of 
their examination grades (e.g., possibly tiie final exam). This would require a meeting with a 
course instructor in order to identify tiie project and tiie criteria and standards for evaluating it. 




Some projects possibly could be dove-tailed with forum activities. Ideas for specific projects 
were cited. 

(5) The course, as it stands, is too crowded. Some of the content from each module that does not 
interconnect with the content of the other modules needs to be eliminated. The points of 
interconnection to focus on need to be determined collectively. Therefore, faculty must spend 
some time planning together. 

(6) The e-mail aspect of the course gets students to talk amongst themselves and to faculty; it gets 
them "involved" and that is an important goal. However, students must monitor e-mail 
postings frequently and with regularity. 

Two of the instructors felt that it might be more desirable to have one as opposed to several 
instructors for this course. Reasons cited included the following: (a) Perhaps so many different teaching 
styles were jarring to the students; and (b) Perhaps the degree of difficulty associated with the science 
modules is such that science "experts" are not needed to deliver the instruction. The suggestion was made 
that if the course had but one instructor, experts could still come in for occasional panel presentations or 
question and answer type sessions where the class would "interview" the expert. 

Pft^^f.rniirse Interviews 

Both of the wrap-around unit faculty and the graduate assistant (collectively referred to as 
instructors in what foUows) completed a post:Course interview. As explained in the methodology section 
above, these interviews posed questions similar to those that comprised the post-course interview for 
students (Table 2). A summary of the instructors responses are categori'--' >^low. In some instances, 
only the two wrap-around unit instructors were asked the question, and accordingly, reference is made to 
just two or the "unit" instructors in describing the findings. 

Major Strengths of Course. Each of the instructors made reference to the thr;.e modules and 
the respective instructors as strength. One instructor emphasized that the students had been exposed to 
three prominent and complicated STS issues from a strong science basis. Anothrr saw as a major strength 
the interrelationship that developed naturally among the three modules as the course proceeded. The third 
instructor saw the students being exposed to three different science faculty members within the context of 
STS issues and their perspectives on the issues as a prominent strength. 

The two wrap-around unit instructors pointed to the graduate assistant as the major strength, 
referring to her as the person who held the course together, who was sensitive to the needs of the students, 
and who met those needs on a daily basis. The forums were noted as a strength by one instructor, given 
they required students to synthesize information, including science concepts, and various views on the 
issue of focus. 

Major Limitations of Course. The wrap-around unit, lack of a strong professor-in-charge 
image, large number of instructors, low number of students, course schedule, labs, and different 
standards across the modules were listed by the three instructors as limitations. Two of the instructors did 
not believe the wrap-round unit served its intended purpose of tying the modules together, possibly 
because some of the time that was to be allotted to the unit had to h-. used for the modules. Similarly, they 
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did not believe a strong image was presented for the students of one percon being in charge of the course. 
One instructor noted the number of instructors as a limitation in terms of students having to adjust to a new 
instruaor every few weeks. Another also noted this as a limitation in terms of establishing consistent 
expectations of students and course policies. The continuity experienced with a single instructor was 
missing. 

One of the instructors was concerned that as much was not going to be learned from this pilot 
offering of the course given the small non-representative or atypical sample of students who registered. 
The degree to which students were required to prepared for the forums and did prepare for the forums was 
noted as a limitation by an instructor. Finally, two instructors described the final exam as approaching a 
level of assessment that required students to ^ply knowledge they developed in the course in ways that 
required higher-order thinking. However, they both did not believe it separated out those students who 
were able to meet the objectives of the course ftx)m those who could not— it was a good first approach, but 
needed revision. 

Suggested Changes. A larger number of suggested changes were noted than limitations. Two 
of the instructors suggested that the course be scheduled in a more traditional format (e.g., two 50 minute 
sessions and one two-hour laboratory period). Two also suggested the need to make better use of e-mail, 
such as by establishing small groups of students that correspond with each other and greater involvement 
by the instructors in reacting to student comments and sharing their comments on e-mail. 

The following suggestions were made by one of the instructors: (a) The grading system and other 
exceptions need to be standardized across modules/units and clearly articulated for the students; 
(b) Linkages that the instructors desire students to see across the modules need to be identified and overtly 
articulated in the instruction and booklets; (c) Students should begin work on the forum activities, possibly 
in the form of an individual or group position paper, half way through a module; (d) A course attendance 
policy is needed that encourages and positively support class attendance; and (e) To the degree possible, 
greater emphasis needs to t-e placed on helping students develop a qualitative understanding of quantitative 
relationships in the course lecture, booklets, and laboratories. 

Absences. One instructor described the high number of absences by a certain portion of the 
class (about 50%) as, "disappointing and somewhat baffling." Another used the word, "puzzling." All 
three saw it related to the length (2 3 hours) and days (Monday and Friday) the course met Two 
instructors indicated that labeling, the course as "experimental" may have influenced the absence rate. All 
three recommended that an attendance policy be instituted. 

Required Work. All three of the instructors believed the number of assignments and amount of 
work requiiwi of the students was about right. They recognized that two to three students complained, but 
attributed the complaints to emotional reactions, absences from class, and laziness on the part of a few 
students. Two of the instructors indicated that they had talked with students who openly repressed 
con-nlaints. One instructor recoitimended adding to the course several intensive writing assignments in 
which students thought about connections across/among the issues of GW, EO, and BC. 



E-MaiL The instructors' responses were diverse as regards e-mail as a course component. One 
instructor heavily endorsed the use of e-mail, though indicated that it was not used to its full advantage by 
the students, instructors, and within the course. Another was neutral on e-mail's value and just indicated 
that it was not used to its full advantage in the course. The third instructor stated that he saw the use of e- 
mail as an interference in the course and was against it. 

Exams. All three instructors perceived the exams were fair. Two of the instructors noted that the 
exams after each of the three modules were structured differently because they were prepared by different 
instructors. In recommending that they should have been more similar in format, they implied that this 
might have affected student performance. Two of the instructors expressed pleasure that the final exam 
involved students in learning activities (e.g., analysis of written material), rather than simple recall of 
information. 

Forums. All three instructors felt the forums were a great idea in terms of having the student 
approach the issues from a societal perspective. Also, one instructor noted tliat it helped prepare the 
students to speak publicly on issues. Each recommended having the students prepare in advance for them. 
The degree of that advance participation varied. One instructor recommended having the student begin a 
position paper that would serve as a basis for their forum testimony at the mid-point of each module. 
Another instructor noted that the groups should have been estabUshed sufficiently in advance for the 
student to be able to meet to prepare their position statement One instructor recommended that the 
graduate assistant or professor-in-charge run the forums in the future so there is continuity. 

Instructor Coordination. One instructor rated the coordination as very good: 

We had planning meetings and people got along well. 

This instructor indicated, however, that from a logistical point of view during delivery of the course that 
the coordination was weaker, given that one nx)dule instructor was replaced and another recently had been 
promoted to an administrative position. Additionally, the two wrap-around unit instructors were not 
available as frequently as was anticipated. 

A second instructor indicated that more was needed— that each instructor did his thing in isolation. 
This instructor suggested that at least there needed to be nwrc references to up-coming and formerly 
covered connections in the other modules and the unit, and in the booklets. He expressed concern that the 
writing editor for the module and unit booklets never had all four booklets at once to look for the 
connections. 

The third instructor indicated that it needed to be made clear to all instructors what the policies 
should be across all of the modules— the "nuts and bolts stuff'. This instructor felt the transition between 
instructors went well because the instructors made an effort to attend other class sessions. 

Laboratory Activities and Reports. One of the instructors felt he had insufficient 
information to critique the laboratory activities and the two others were basically pleased. In particular the 
field trips and computer-based laboratory sessions were noted as both educationally valuable and highly 
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interesting by one instrjctor. This instructor also suggested that what constitutes a lab in this type of 
course needed to be more clearly defined for the students early in the course, given some students 
apparently raised questions. 

Lecture. One instructor described these as being too long. A second felt that lectures may not be 
conducive to this type of course and that perhaps tiie instructors felt tiie student could get into the issues 
faster than they actually were able to. A thiid instructw had this to say about the lectures: 

[They were] probably tiie single best done part of the course. Each [instructor] had a different 
style. My impression was tiiis was not a problem, in fact woriced to our advantage in terms of 
variety. 

Natural Science Credit for the Course. None of the instructors questioned that the course 
deserved to be recognized as a general studies course for natural science credit One instructor made direct 
reference to the course module exams and tiie final exam as evidence tiiat tiie enrolled students learned 
science concepts at an appropriate level of rigor for a general studies science course. 

Number of Instructors. One instructor indicated that having separate instructors with 
expertise in the area they taught was a great opportunity for students, especially in a 200-level course. 
Each of the tiiree instructors still appeared to feel that there were too many instructors in terms of course 
continuity and tiiat it would be difficult to justify die number beyond the course pilot period. Two 
instructors recommended one for each module, witii one serving as die professor-in-charge, and one 
graduate assistant for the wrap-around unit. 

Prerequisites. One instructor noted that it was not appropriate to think about prerequisites for a 
general studies course— that is, by university policy, tiie course has to be taught so students can pick up 
any so called prerequisite concepts in tiie course. Anotiier continued tiiis idea: 

In a course like tiiis, you have to accept tiiat students come from very different backgrounds, and 
you have to be willing to adjust. 

Reading Material. Although one instructor felt they were pretty good as a first edition, two 
otiiers expressed concerns tiiat (a) connections across units need to be built into the booklets and (b) that 
probably students did not read tiiem. 

Course Scheduling. All three of the instructors preferred moving to a tiirec-days per week 
schedule. One instructor observed the following: 

[T]he students got bored and the instructors digressed; neither were use to the long periods. 

Graduate Assistant's Role and Functioning. The graduate assistant felt she was respected 
as a member of the team and had tiie freedom to contribute as she saw fit. The wrap-around unit 
instructors confirmed this notion: 

Witiiout K , tiie course would have been a flop. 
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She could have taught it all, but she is an unusual person. 
She was able to take on more than we can expect. 
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Understandings Gained, The instructors focused on the students gaining a broader 
perspective on the issues dealt with in the course. One instructor noted: 

By the end of the first forum the students were very aware of three things: Simple policy options, 
such as those noted in the paper, are too simple to be workable; These options arc backed by 
various groups, not necessarily the good and bad guys, many of whom had our best good in mind; 
and These are very complicat&d issues. 

Similar views were expressed with respect to the GW module by another instructor who said the 
students learned this: 

[Tlhat it is not global warming. That it is global climate change. . . .They learned about the 
Umitations of our knowledge and modeling, and that the media presentation is over-simplitied. 

With regard to the EO module, this instructor felt the learning was more focused on the amount of 
energy we waste and carb(.n dioxide we put into the air. The energy audit lab was pointed to as a key 
activity here. With regani to student learning in the wrap-around unit, one of the instructors indicated that 
students learned most about the United Nations system, given that was a main emphasis, and that there 
was no direct science learning here because that was not the focus of the wrap-around unit 

Important Interrelationships. One instructor pointed to the realization that there are 
interrelationships among the issues dealt with in the course, as the most important interrelationship 
learned. The second instructor saw EO as the key module, and interrelationships among the wrap-around 
units as being very important Both instructors referred to the final exam results as indicating the smdents 
were aware of primary interrelationships among the three issues at the end of the course: 

mhe smdents could explain the connections among the modules at least at a surface level, such as 
the impact of X on global warming and biodiversity, and some basic ecological concepts, and how 
waste impacts these. 

Preparation for TesHmony. The instructors did not have a preference as to which issue the 
students elected to testify about anticipating that one would have provided an entry into the other two 
because of interrelationships. One anticipated that more students had selected EO, given energy appeared 
to be the most central issue to him. The other instructor anticipated that GW was selected by more 
students with BC second given these are more glamorous than EO. 

The instructors' responses on how they anticipated the students would prepare for the testimony 
were different. One hoped they foUowed a model something Uke the forum: (a) they suinmarized what we 
know and don't know about the issue, (b) laid out the two to four options we have, and (c) picked one 
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option and argued for it, especially as it has implications far other options, thus expressing an 
understanding (pros and cons) of different options. This instructor felt the booklets would have been 
good resources for the students to use, as would the Race to Save tks Planet (Anncnberg/CPB Collection, 
1990) video series, e-mail gophers, and course handouts. This instructor inferred that utilization of the 
library as a resource was something that the course should have emphasized more so. 

The other instructor would like to have seen the student start at the library: (a) find out the 
positions of various groups on the issue and why the position is held by going to references found in 
Readers' Guide; (b) look at the arguments made by each group and formulate their own position; and (c) 
prepare notes for the testimony in which they support their position while arguing against other positions. 
This instructor saw the previous as something the students should already know— "a first cut of Research 
101." Still, the instructor added: 

We should have had training sessions on this, along with preparatory assignments in each forum. 
I anticipate that their actual level of performance was weak. 

Effect on Students' Interests, Action and Lifestyles. The instructors indicated that the 
students came to the course with interest in STS issues, particularly one or more of the environmentally 
related issues focused on in the course. Still, they were not sure exactly how the course might have 
impacted students' interests in STS, actions on STS issues, or lifestyles. Yet, they hoped the students had 
moved from the doom and gloom perspective they saw as typical among undergraduates and the general 
public. One instructor commented on the course's focus on these outcomes by saying: 

Not very much; the piece that was missing is they did not get hands on experience with actions, 
only through videos. The forums need to extend students to action taking groups. 

Additiona l Haffl Sources 

In addition to the interviews with the students, instructors and graduate assistant, the evaluators 
administered a 16 item VOSTS instrument pre and post to student subjects, made informal class 
observations, reviewed the course booklets, and examined the course exams. 

The 16 item VOSTS instrument was the same one used by Schoneweg (Schoneweg & Rubba, 
1993) to assess college level students' understandings of STS interactions. Six students completed both 
the pre- and post-test. The mean score on the pre-test was 36.00 and 36.17 on the post-test, out of a 
possible 48 points. While tliese were not found to differ significantly, the scores arc similar to the pre- 
instruction scores of the students in the introductory STS and physics (non-majors) courses in the study 
by Schoneweg. That is, these six students initially held views on the interactions among science, 
technology, and society that expressed a number of legitimate points, but were not considered fully 
appropriate or legitimate. Additionally, these limited views did not apix;ar to change over the period of the 
semester. 



2j 

27 



Collectively, the two course evaluators attended the majority (21) of the individual class sessions 
Oectures, laboratories, and forums); some general observations follow. These and otiier class session 
observations were utiUzed in concert with student and instructor interview responses to foraiulate 
assertions and recommendations for future offerings of STS 251 . 

For the majority of the lectures and fonims, more tiian one of the course instructors were in 
attendance and participated to varying degrees in class discussion. The module instructors appeared 
sensitive to the complexity of their topics relative to the science backgrounds of the students: Material 
usuaUy was presented slowly and tiie instnictors regularly asked students if tiiey had any questions. 
Additionally, instnictois attempted to disclose students' prior understandings of various science concepts 
through classnwm dialogue and in one case, a prc-test The amount of teacher-student discourse during 
lecture varied across tiie modules, but overall was beUevcd to be considerably more than what would have 
occurred in the vast majority of otiier undergraduate courses taken to fulfiU a portion of the university's 
natural science requirements. AdditionaUy, tiie fomm sessions provided ample opportunity for and 
resulted in considerable dialogue amongst tiie students: They placed students in various scenarios and 
pix)vided focus questions for small group and class discussion. The instructors also were sensitive to 
students' interests and allowed students' requests and questions to direct portions of class lecture. 

Course attendance began to drop quite a bit by the end of tiie first module and students often came 
late to class. Mid- way tiirough tiie course, tiie lead instnictor discussed tiiis attendance problem in lecture 
and sent a related memorandum to each student, encouraging future attendance. Course attendance 
subsequently increased to some extent. 

/^>;gprtinns and Rppnmm«>ndations 
The course evaluators posed tiie following 12 assertions and corresponding recommendations 
concerning STS 25 1 , based upon tiieir interpretation of tiie evaluation data. Readers of tiiis paper may 
believe tiie evaluation data reported herein supports assertions and recommendations in addition to or 
alternative to tiiose presented. Recognizing tiie limitations of all evaluation processes and pnxedures, the 
evaluators have made an attempt not to overextend tiie data, and so have limited tiie assertions and 
recommendations to tiiose tiiey beUeve are strongly supported by tiie data. 

The presentation of some of tiiese assertions and recommendations should not be misinterpreted as 
a condemnation of STS 251, but ratiier us constnictive review. Even tiiough tiie evaluators identified 
certain Umitations associated witii STS 251 during tiiis first offering, tiie strengtiis of tiie course far 
outweighed tiie shortcomings. Overall, tiie students indicated tiiat tiiey found tiie course to be interesting, 
worthwhile, relevant, and useful. Beneath tiie caution apparent in tiie points expressed by the instnictors, 
tiie evaluators perceived tiiat tiiey were pleased witii tiie first offering of tiie course. Additionally, tiie 
evaluators perceived a great and genuine interest on tiie part of tiie course developersAnstiuctors to 
capitalize on the course's strengths, limit the weaknesses, and test new ideas in future offerings. 
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Asserfion 1— .roiiriee F ocus. Contpnt. nnd Marketing 

The focus and content of the course were exemplary relative to the kinds of courses this or any 
post-secondary academic institution should offer as a means for students, as citizens of a global and 
informational society, to fulfill undergraduate general studies natural science credit requirements. The 
course provided content from a breadth of science disciplines (e.g., biology, earth science, and physics); 
interrelated science content across the disciplines and to technology and major societal issues that arc 
global in scale; and attempted to empower students towards taking citizenship action to resolve these major 
societal issues. The science content was contextualizcd in real world problems: The three STS issues 
focused upon in the course— GW, EO, and BC— are critical issues facing hunoankind and hold great 
interest for the potential STS 251 student population. Students emerged imm the course with a sense of 
the importance of what has been learned. Overall the course was of considerable value to the majority of 
the students. However, course enrollment was low for this first offering partly because course advertising 
did not capitalize sufficiently on the students' interest and ccmcems about "the environment." 
Rppnininendation la 

Retain and aggressively market the course as a highly viable option for students to use in fulfilling 
a portion of their undergraduate general studies natural science credit requirements at this institution. 
Recommendation lb 

All course ads should take advantage of connections that can be made between course content and 
the potential/target audience's concern for and interests in environmental issues, especially as these relate 
to lifestyle decisions. The word "environment" needs to be emphasized in all course mariceting. 
Assprtion ? — Course Obiectives 

The STS 251 Course Objectives (Figure 1) were appropriate for a college general studies STS 
based science course and the intended student population. However, the instruction provided in the course 
focused mainly on the lower order cognitive objectives (la through c). The higher order objectives. Id, 2a 
through c, and 3a through c, were not addressed sufficiently in the course instruction. Relative to 
objectives la through c, students perceived that the course enhanced especially their understandings of the 
mechanisms underlying the greenhouse effect and GW, and the concept of energy efficiency. 
Furthermore, the student interviews revealed that students emerged from the course with an understanding 
of the "hidden or real cost" of EO and viewed the following as a key interrelationship amongst the three 
course modules: Energy consumption is central to and drives GW and problems associated with 
biodiversity. In regards to the objective (Id) on reaching informed decisions and taking positive 
citizenship actions on course related issues, students believed "energy conservation" to be a key citizenship 
action they and others could take towards the resolution of the related issues, and the course appeared to 
impact positively their desire to take such action on a personal level. However, beyond this, it generally 
was not apparent that the course enhanced the students' research skiUs in investigating issues or related 
decision making and action capabilities (objectives 2b and c). 
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Bffftpmpndfliion 2 

The basic factual information on each of the issues can be presented in the course booklets. The 
lecture/discussion portion of the course should go beyond and extend the material dealt with in the 
booklets. Instructional strategies that involve students in higher order learning (e.g., cooperative groups, 
debates, simulations) should be used extensively. Use of instructional strategies tiiat facUitate higher order 
thinking likely would realize tiie additional benefits of increasing the intellectual excitement of the course 
and reducing absenteeism. Two additional suggestions for strategies and assignments that address the 
higher order course objectives of Id and 2a through c are described below. These suggestions arc in the 
form of "general ideas" and are not intended to provide complete descriptions. If implemented in some 
fashion, these strategies/assignments simulianeously should encourage student interests as identified in 
course objectives 3a tiirough c; engage students moreso iii the learning process; facUitate "transfer" to other 
contexts and appUcation of what is being learned; and reduce absenteeism. Furthermore, tiiese 
assignments each result in evidence that can be used along witii course tests and lab reports to determine 
students' course grades. They would provide alternative and auUientic means for assessing students and 
their inclusion would respond positively to the nationwide call to reform science assessment Each project 
also may be interfaced with die class forums. 

Suggested Assignment 1. Students keep a "responsive" journal and turn it in weekly or bi- 
weekly. Herein, each student responds to a question posed weekly by an instructor, which is related to 
one or more of these course objectives and that requires students to reflect, think critically, and do a little 
research. Students should be encouraged to make other entries in the journal, such as reflections on 
course content and related current events. 

Suggested Assignment 2. Early in the course, each student identifies some type of project 
he/she wUl complete by the end of the course that has the potential to demonstrate competence has been 
gained relative to tiiese objectives. Two or three students could collaborate on a project, if it appears 
constructive. The project must be "worked out" witii and verified as acceptable by tiie instructor. 
Subsequendy, a succinct description of the outcomes of tiie project and criteria for assessing these 
outcomes must be put in writing by the student and submitted to die instructor. Brief progress and 
terminal oral reports on each project could be given by each student (or group) midway Uirough and at the 
end of the course, respectively. 
Assprtinn .^ —Tnurse T.eadership 

The students were unclear about who was die course leader and were not presented with a "united 

front" in terms of course policies. 
RpcnminenHatinn 3 

One faculty member needs to be identified to the students and function as die profcssor-in-charge 
of the course. The joint development and application of consistent policies (e.g., attendance, assessment 
and grading) across die modules/units should come under diis individual's leadership. 
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Assprtions 4~rotirse Instructors 

The quality and variety of instructors, including the graduate teaching assistant, enriched the course 
and were mapr strengths. Beyond their content expertise, they were concerned about the quality of their 
teaching, were receptive to students input, and showed a genuine interest in teaching this general studies 
course. 

Rprnmmpndation 4 

Retain four instructors, one as the "expen" for each module and one to conduct the wrap-around 
unit If the present instructor of the GW Module is not available for future offering of the course, the 
present GA has the background and expertise, plus the teaching skills, to serve as an instructor for the GW 
Module. One of the four instructors also must serve as the professor-in-charge. 
Assertion — Interrelationships Across the Course 

Interrelationships across course content, including those necessary to make the transition from one 
module to the next, were not made to the extent that the course faculty desired and anticipated that they 
would be made. For the most part, the students developed a relatively unsophisticated perspective on the 
interconnections across the three science and technology issues— that energy consumption is central and 
drives GW and problems associated with biodiversity. 

Recommendation 5a 

Faculty need to spend time in advance of the next course offering planning for the more extensive 
coverage and integration of course interrelationships. The faculty should decide upon specific important 
interrelationships that can be carried through all modules. The outcomes and concepts identified in facult>' 
interviews and reported herein previously, and the concept maps used by two of the three faculty in their 
components of the course, may be useful in this endeavor. 

Recommendajtion Sb 

Each course booklet needs to make visible and emphasize the important interrelationships identified 
through the faculty planning activity recommended above. 
Recommendation Sc 

The first day of class should devote substantial time to setting forth a rationale for why these three 
science and technology issues (GW, EO, and BC) were chosen (as opposed to other issues) and the 
various interrelationships that cut across these three issues. 

Recommendation Sd 

The wrap-around unit needs to emphasize moreso the interrelationships across the three science 
and technology modules. However, one should not expect that the wrap-around unit alOBS. will serve as 
the "glue" that holds the three science and technology modules together and elucidates the 
interrelationships among the three issues. Class introductions and ending statements should be used that 
provide a type of course "road map" for students. 



Additional learning activities and assignnients (e.g., concept mapping) should be added to the 
course to facUitate student understanding of the important interrelationships. Also, these interrelationships 
should be focused upon in student assessments. 
Assprtion ^rontcnt Dgnsitv 

The EO module, and to some extent the GW module, were too information packed. The 
instructors of these modules appeared not to have sufficient time to do much more than "cover" the 
material. This, combined with a quantitative emphasis, made the EO module somewhat overwhelming to 
the type of students who are anticipated to comprise the future enroUment of the course. 

Rppommen dfltion 6a 

More dialogue and discussion that is directly related to course interrelationships and the, higher 
order course objectives, needs to be initiated with the students during the lectures and labs. 
jRppnmmen dation 6b 

The set of concepts that are most critical to students' understanding of EO need to be identified, 
and the module reworked to focus on these. Though not necessary, such a review also is recommended 
for the other two modules and the wrap-around unit. Identifications of conceptual linkages across the four 
modules/unit would be facilitated if aU instructors participated in these reviews. 

BpfpmmPndation 6c 

To a greater extent and where possible, quantitatively based concepts in the EO and GW modules 
need to be approached qualitatively. The very successful physics textbook. Conceptual Physics (Hewitt, 
1993) is an exemplar of how this might be done. 
Assertion 7— I.<>pture«i anri Booklets 

The course lectures generally covered the same material as could be obtained from the course 
booklets. Because many students believed it was not necessary to both read the booklets and attend 
lecture, tills negatively affected course attendance. It also resulted in some students not reading the 
booklets. Additionally, it compromised the engagement of students in criticaVthought-provoking activities 
during class time. 

As stated in tiie recommendation for Assertion 2-The basic factual information on each of the 
issues can be presented in the course booklets. The lecture/discussion portion of the course should go 
beyond and extend the material dealt with in tiie booklets Instructional strategies that involve students in 
high order learning (e.g., cooperative groups, debates, simulations) should be used extensively. Use of 
instructional strategies tiiat facUitate higher order thinking likely would realize the additional benefits of 
increasing tiie intellectual excitement of the course and reducing absenteeism. Otiier recommendations 
made in this report also speak to the lectures and booklets. 
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Assertion 8 — The Lahnratorv 

Overall, the laboratory activities were meaningful to the students, and to some extent, personally 
applicable to their lives. However, the amount of out of class woric associated with one of the GW labs 
was perceived as being too much work all at once . 

Repommendfltion 8 

Spread out and/or streamline the amount of out-of-class work associated with the GW labs. 
Throughout the unit and course, consider adding more or modifying existing labs to apply or reveal 
student lifestyle, especially computer-based modeling and ones with environmental implications. A 
computer-based simulation for the emissions lab and a new lab on land use, which were recommended by 
instructors, should be seriously considered. 
Assertion 9 -— The Forums 

The forums were a ma jor strength of the course in that they encouraged application of 
understandings developed in die respective modules to related real world problems and provided an 
opportunity for students to express their views. Nonetiieless, student attendance at the forum sessions 
was compromised by not routinely making some assignment to the students in advance as necessary 
preparation for and to be utilized during and in follow-up to each fomm. 

Repommendation 9a 

For every forum, establish small groups and provide a small group assignment in advance (e.g., at 
the beginning or mid- way through die previous science and technology module) that is to be 
utilized/delivered at the forum and in a follow-up activity. An example assignment could be the 
preparation of a position paper or testimony that sets forth and supports a pre-assigned stance on the issue 
targeted by the forum. As foUow-up, students might suggest and explore the viability of certain individual 
and group actions. The forum assignments and related presentations should be considered as important 
evidence of student learning, and accordingly, be utilized in die assessment of students and determination 

of grades. 

Recommendation 9b 

AU of the forums should be led by the same faculty member. The professor-in-charge of the 
course is the most likely candidate. 
Assertion 10— Cours e Organization 

The course syUabus delivered on the first day of class was not sufficientiy comprehensive and clear 
in terms of die "day to day" course content, related dates for assignments and exams, and how grades 
would be determined. 

Bffpmmendation 10 

One pomprehensive course syllabus should be provided to all students at the first class session, as 
opposed to separate syllabus components for each of the diree modules/unit distributed at different times 
over the course of the semester. The course syUabus also should discuss the unique nature and aspects of 
die course and describe how grades will be determined in clear detail. 



Assertion ll—Tnu.sp Srhpdiiling 

Course enrollment was affected negatively by the days and time blocks within which the course 

was scheduled. 

Bffpmmpndfltion 11 

The course should be scheduled as two 50 minute lecture/discussion periods and one 100 minute 
lab period per week. If more instructional time is needed the course should be made four credits. 
Assprtinn 1 ?— ITso nf F.Mail 

As used, e-mail made little contribution to the course. 

Bpfftpmpnrifltion 12 

The purpose of using e-mail needs to be clearly defined and e-mail should be implemented in 
accordance with that purpose and the power of this technology. The instructors should initially focus on 
using e-mail for between class communications among the students and instructors, and the exchange of 
views on aspects of the STS issue under study. This use should be implemented if and only if: (a) all 
students receive training in the use of e-mail, and (b) all members of the faculty and class agree to read and 
respond to their e-mail on some specific regular basis. Once an e-mail communications Unk is established 
and functional, it may be possible to use e-mail for other related purposes (e.g., as a support for groups 
work, to locate resourses on the university's electronic information access system and the Internet). 
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After course completion, students will possess greater knowledge and understanding of: 

a. science facts, concepts, and processes central to GW, EO, biodiversity, and their 
interrelationships. 

EM Evaluators interview volunteer students pre and post each module, and at the end 
of the course. 

Concept mapping exercise completed by volunteer students prior to the first and 
at the end of each module. 

Evaluators have access to student exams. 

b. technology as it iinpacts GW, EO, biodiversity, and their interrelationships. 

EM Evaluators interview volunteer students pre and post each module, and at the end 
of the course. 

Concept mapping exercise completed by volunteer students prior to the first and 
at the end of each module. 

Evaluators have access to student exams. 

c. key social science and public policy concepts that underlie decision making about GW, 
EO, biodiversity, and their interrelationships. 

EM Evaluators interview volunteer students pre and post each module, and at the end 
of the course. 

Concept mapping exercise completed by volunteer students prior to the first and 
at the end of each module. 

Evaluators have access to student exarns. 

d resources and methods for investigating the science, the technology, and the society 
issues related to the three course topics (i.e., GW, EO, and biodiversity) and their 
interrelationships. 

EM Evaluators interview volunteer students at the beginning and end of each module 
and at the end of the course, placing them in simulated issue investigation 



(Continued) 
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After course completion, students will possess greater skills: 

a. in scientific and technological literacy. 

(This primary skill outcome, which is presupposed by the others, can be measured by 
assessing student ability to read and comprehend science articles in both the general press 
such as the New York Times or Time and, for the specific module issues, general 
audience scientific publications such as Sriftntific American or New Scientist.) 

EM Evaluators have access to course final exam, which will include a technical 
article. (This is a very limited definition of scientific literacy.) 

b. in pursuing knowledge about GW, EO, and biodiversity issues. 

EM Evaluators interview volunteer students at the beginning and end of the course, 
placing them in simulated issue investigation. 

c. in reaching informed decisions and taking positive citizenship actions that can be taken 
toward the resolution of public issues related to GW, EO, and biodiversity and to STS 
issues in general. 

EM Evaluators interview volunteer students at the beginning and end of the course, 
placing them in simulated issue investigation. 

After course completion, students will possess greater interet: 

a. in reading about other science- and technology-related issues. 

b. in taking other STS or related courses. 

c. in becoming more informed for the sake of getting involved. 

EM Evaluators interview volunteer students at the beginning and end of the course, 
placing them in simulated issue investigation. 

Evaluators administer 16 VOSTS items at the beginning and end of the course. 



Figure 1 (continued). STS 251 Course Objectives and Corresponding Eva/im/ion Means (EM) 
that Comprised the Evaluation Plan Conceived by the Course Evaluators 
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Figure 2. Concept Map Prepared by Instructor for GW Module 
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rnncftpt Map Assignment #1 



Below please find a list of terms which you should use to construct one concept map. In addition tP the 
tftrms shown, please identify three other s which vou want tn include in this man. Map these three 
additional terms along with those provided in the list below. You do not have to select these three 
additional terms before starting the map. It is likely that other terms will come to your ^d as you are 
constructing the m^. As these other terms come to mind, select the three that you believt are the most 
important and map them in. 

The map should illustrate vonr current im rierstandinf of the relationships between these terms. So, we 
ask you, please: 

a) do not consult with anyone else to help you prepare the map; and 

b) do not "research" further any of these terms before or while you prepare the map. 

The map is due on Monday, 1/17, at the start of class. You will be paid a total of $7.50 for 
completing the self instruction packet plus this fu^t concept map assignment 



List of Terms: 

Climate Feedback 
Environmental Issue 
Environmental Law 
Environmental Policy 
Food Production 
Fossil Fuels 
Global Climate Models 
Global Radiatoin Budget 
Greenhouse Gases 
Greenhouse Warming 
Land Use 
STS Issue 

Figure 3. First Concept Map "Assignment" for Students 
to Complete as Part of Course Evaluation. 
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Knowledge and Awareness 

Greenhouse warming as international problem. 

What is being proposed to conserve species. 



Understandings 

Greenhouse warming, related physical problems, and social factors that drive as well as 

constrain solutions of Greenhouse warming. 
Energy: forms; types of units; laws of conversion; limits on resources; environmental 

impact associated with conversion; alternative options (soft and hard paths) for future 

energy policy. 

Fundamentals of ecosystems, species diversity, evolution. 
Reasons underlying species extinction. 

Social, political, and economic context in which biologists work to achieve goals. 
Perceptions 

A change in perception of Global Qimate Change as a problem. 



Skillft/Applications 

Communicate to others tiiat Global Climate Change is a problem. 
Distinguish between the different forms of energy and interconvert units of energy. 
Read (and comprehend) articles on biodiversity beyond die newspaper level (e.g., in 
Scientific American). 

Appreciation 

Complexity of our climate system and models employed to predict climate change. 

Figure 4. Outcomes Each Instructor Desired Students to Achieve 
as a Result of Completing His/her Respective Module. 
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niohal Warming Module 
greenhouse warming 
climate feedback 

changes in global radiation budget 
emission of greenhouse gases 
global climate models 
global climate modules reliability 

Energy Oprionf; Module 
energy 

energy conversion 

entrc^y 

efficiency 

exponential growth 

power 

work 

pollution 

Biodiversit y Cff^^^^^Q" Module 
nutrient cycling 
energy flow 
continuity 

discrete boundaries/bounded entities 
space and area (as limiting factors) 
primary productivity 
levels of consumerism 



Figure 5. Concepts Deemed by Instructors as Most Central to Each Module 
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Table 1 

Questions Comprising Pre- Module Interviews with Students 



1) What influenced your decision to enroll in this course? 

2) Thinking about the topics and modules of study in this course, what content is of the greatest 
interest to you? 

3a) What would you most like to get out of this coursc-in other words, what are important course 
outcomes for you? 

3b) (dtr. each outcome provid ed in response to 3a) is important to me because: 

4a) What, if any, relationships do you sec amongst the three naodules-Global Warming, Energy 
Options for the Future, and Biodiversity Conservation-that comprise this course? 

4b) In what way(s) is the wrap-around unit on Science, Technology, and Environmenal Policy 
related to the three modules? 

5a) What do you believe are the two most important relationships shown in your concept map? 
5b) What is it about (cite each relationship prn viHrH in response 5a) that makes it so important? 

6) In this map, you have included three terais which you believed were important but were not 
provided on the list of terms for this assignment. These terms are (cite tgims)- For what reasons 
did you choose to include each 6f these three terms? 

7) (The smdent was asked what thoughts and ideas came to mind 'hen the student thought about 
each of two concepts, e.g., " climate feedback" and "global radiation budget", that the module 
instructor had identified as being amongst those most central to the module.) 

8) (The student was asked what thoughts and ideas came to mind when the student thought about 
each of two concepts, e.g., "efficiency" and "exponential growth", that the module instructor had 
identified as being amongst those most interrelated with the other course modules.) 

9) (The student was asked what thoughts and ideas came to mind when the student thought about 
each of two concepts that the wrap-around unit instructors had identified as being amongst those 
most central to the wrap-around unit. These concepts were "environmental policy" and 
"environmental issue") 

10) (The student was asked if they believed that the issue presented by the upcoming module of 
study, e.g., GW, was a threat and to provide reasons for their answer) 

1 1) (Students were presented with a scenario where they were to walk the interviewer through the 
preparation and deliveiy of a testimony on the critical issue targeted by the module of focus, e.g., 
"Energy Options for the Future: Is it a concern? What should be done?" Refer to Table 2, 
question 6, for the script followed in presenting this scenario.) 

12) Is there anything else you would like to tell me about the course STS 25 1 at this point? 
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Table 2 

Questions Comprising End-of -Course Interview with Students 
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1) What do you pett^eivc to be the major strengths of the course? (Please explain.) 

2) If vou had the power to change the course, what changes, if any, would you make? For any 
change you mention, I would also like to know your reason for suggesting the change. 

3) I would Uke your comments on various aspects of the course: 
Scheduling 

Number of instructors 
Lecture 

Reading material 

Forums 

Lab 

Lab reports 
Exams 

Amount of work 

E-mail 

Absences 

Course prerequisites 

In your opinion, should this course count as a general studies natural science course? Why? 

4) Prior to taking this course, each student probably had some prior knowledge and undersmnding 
of the course content. I would like you to consider ym. prior knowledge and understanding of 
the content in this course. Specifically, I would like you to teU me: What do you taiow and 
understand WSL that you did not know and understand prior to taking this course? Lets start 
with... 

Global Warming 
Energy Options 
Biodiversity 
Public policy 

5) What do you consider to be very important interrelationships among all three course topics of 
GW, EO for the Future, and BC? (Describe how is interrelated.) 

€i Now, I have a scenario I am going to describe. Let's imagine it is the real thing Suppo^ the 
University Student Government and Eco- Action were jointiy sponsonng three different Forums 
in the future. The topics for these three different forums arc GW, EO for the Future, and BC. At 
each forum, testimonies will be presented by concerned individuals. 

In a testimony, the concerned individual sets forth and substantiates his/her views and position 
on the topic of concern. Testimonies on various topics of concern and on proposed laws are 
often given in front of Congressional Committees. 

(Continued) 



(Tabic 2, continued) 



You have been selected at random from this class to give a testimony at one of these three 
forums on the topic of concern: GW, EO for the future, or BC. You must choose: Which one of 
the forums will you testify at? Your topic will be (depending on your choice): 

- . Is it a concern? What should be done?" 

a) I would like you to walk me through your preparation for this testimony. Please tell me 
how you would go about preparing to give this testimony. 

b) In your preparation, suppose you ran across one or more resources which presented a 

view on tiiat was contrary to your position. How, if at all, would you use 

such resources? (What are your reasons?) 

c) Let's suppose that you have completed your preparation for this testimony and that your 
present knowledge on GW is what you have learned through this preparation. In 
presenting your testimony, what would be the key points you would make to your 
audience? 

d) Would you recommend that the members of your audience take actions? (If so, which 
ones and for what reasons?) 

7) For what Tcason(s) did you choose the above testimony topic as opposed to the other two? 

a) Do you feel the topic you chose represents mart of a threat than the other two topics? 
Please explain. 

8) How would your preparation for the testimony hav;. auT^red if you hadJlQl taken STS 251 ? 

9) In what ways, if any, has this course influenced your inxc rest in STS? 

10) Has this course influenced your interest in taking other STS courses or courses related to the 
topics presented in this course? (Please explain.) 

1 1) Has this course influenced your thinking about the resolution of STS issues such as GW, EO for 
the future, of BC? (Please explain.) 

12) Has this course influenced your thinking about your lifestyle? (Please explain.) 

13) Is there anything else you would like to tell me about the course STS 251->anything else you 
believe that would be important for the evaluators of this course to know? 



Table 3 

Questions Comprising Pre-instruction Interview Jor Course Faculty 

1) What outcomes do you desire for STS 251 students in your module? 

2) Tell me about the importance of each of these outconnes. 

3) What is the relationship between the outcomes you nientioned and the STS 25 1 course objectives? 

4a) What are the most important or central concepts/big ideas in your module that you want students 
to understand? 

4b) Of those central concepts/big ideas, which three do you consider to be most interrelated with the 
other modules in this course? 

5) How do you see the relationship/interplay between all of the modules? 

6) What else, if anything, would you like to say--anything you feel that would be important for the 
evaluators of this course to know? 
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Table 4 

Questions Comprising Post-moduk Interview with Module Instructors 



1) At our prc-modulc interview, you identified several outconaes (provide list of outcomes) that you 
desired students would achieve as a result of completing your module. Now Aat you have 
finished teaching this module, please tell me: 

a) Are these (still) the outcomes that you hope students achieved? In other words, what, if 
any changes in this list are necessary to reflect the outcomes that you now hope students 
actually ddd achieve? 

b) To what degree do you believe students actually achieved each of these outcomes? What 
are your reasons? 

2) If you taught this module again: 

a) How, if at all, would you change the list of outcomes that you desired students to 
achieve? 

b) What, if any, changes would you make in your segment of the course? These could be 
changes in the vinritten materials, lab activities, etc. For any changes, I am interested in 
knowing your reason(s). 

3) How do you sec the relationshipAinterplay between all of the course modules? 

4) For the next offering of this course, what recommendations would you make to all of the faculty 
involved? For each recommendation, please give me your reason(s). 

5) Is there anything else you would like to say--anything you believe that is important for the 
evaluators of this course to know? 



